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A  SIMULATION  ANALYSIS  FOR  RANKING  AND  SELECTING  THE 
BEST  COMBINATION  OF  PRODUCTION  AND 
ACCOUNTING  CONTROL  SYSTEMS 

ABSTRACT 

Frequently,  the  objective  of  computer  simulation  experiments  of 
accounting  and  business  systems  is  to  find  the  best  policy,  procedure,  or 
decision  rule.  Previous  accounting  or  business  simulation  studies 
assumed  that  observations  taken  from  each  population  were  normally 
distributed  with  unknown  means  and  known  or  equal  variances. 
Unfortunately,  only  in  rare  cases  can  such  assumptions  be  expected  to 
hold.  This  paper  introduces  a  multiple-ranking  procedure,  which  allows 
for  unknown  and  unequal  variances,  to  analyze  simulations  of  production 
planning  and  accounting  control  systems.  The  model  under  study  is  a 
hypothetical  firm  with  profit  and  sales  as  multiple  objectives.  Two 
multiple  objective  planning  models  with  uncertain  demands  are  formulated, 
and  two  accounting  variance  analysis  techniques  are  used  and  incorporated 
into  the  two  planning  models,  A  two-stage  sampling  procedure  is  used  to 
determine  the  sample  size.  The  simulated  data  are  analyzed  by  a  multiple- 
ranking  procedure  and  then  the  best  policy  with  respect  to  profit  and 
sales  is  selected  separately.  Possible  accounting  and  business 
applications  are  also  mentioned. 


A  multiple-ranking  procedure  is  a  statistical  process  which  controls 
the  specific  probability  of  correctness  in  selecting  the  best 
alternative.  Although  a  fundamental  multiple-ranking  procedure  was 
developed  nearly  thirty  years  ago  by  Bechholfer  [1954],  decision 
scientists  and  accountants  have  shown  little  interest  in  this  technique. 
The  two  major  reasons  are  that  previous  multiple-ranking  procedures:  (1) 
were  developed  primarily  for  scientific  experimentation,  and  (2)  assumed 
that  the  observations  taken  from  each  population  are  normally  distributed 
with  unknown  means  and  known  or  equal  variances,  a  situation  which  rarely 
exists  for  accounting  and  business  situations. 

With  the  advent  of  computer  simulation  techniques,  complex  business 
systems  can  be  studied  via  controlled  experiments.  The  recent  development 
of  multiple-ranking  procedures  with  unknown  and  unequal  variances  by 
Dudewicz  [1972]  and  Dudewicz  and  Dalai  [1975]  provides  a  solution  for 
simulation  experiments  under  the  unknown  and  unequal  variances. 

Oftentimes  managers,  decision  scientists,  and  accountants  face  the 
situation  of  selecting  the  best  alternative  from  among  a  number  of 
policies,  procedures,  or  decision  rules.  Multiple-ranking  procedures 
offer  a  useful  approach  to  this  kind  of  decision.  The  purpose  of  this 
paper  is  to  introduce  a  multiple-ranking  procedure  with  unknown  and 
unequal  variances  and  apply  it  to  analyze  simulations  of  multiple 
objective  production  planning  and  accounting  control  systems. 

The  paper  begins  with  the  background  and  purpose  of  the  study.  The 
second  section  reviews  different  multiple- ranking  procedures  with  various 
assumptions  about  population  variances.  It  is  followed  by  a  description 
of  Dudewicz  and  Dalai's  heteroscedastic  multiple-ranking  procedure.  Then 


an  accounting  system  design  proolem  for  a  multiple  objective  firm  is  u~ed 
as  an  illustration.  This  model  forms  the  basis 

for  computer  simulation  experiments  and  analysis  using  a  multiple-ranking 
procedure  with  unknown  and  unequal  variances.  Finally,  possible 
accounting  and  business  applications  are  presented. 

MULTIPLE-RANKING  PROCEDURES 

In  accounting  and  business  experiments,  frequently  a  decision  to 
select  the  best  policy,  procedure,  or  decision  rule  must  be  made.  For 
example,  given  more  than  two  machines,  the  one  which  will  produce  the 
highest  mean  output  per  year  is  sought;  or,  from  among  more  than  two 
inventory  valuation  methods  being  simulated  on  a  computer,  the  one 
resulting  in  the  highest  mean  profit  is  desired.  A  conventional  F-test, 
that  assumes  that  population  means  are  equal,  does  not  provide  adequate 
information  to  decisionmakers.  For  example,  Conway  [1963,  p.  531]  stated 
that  "...  the  analysis  of  variance  seems  a  completely  inappropriate 
approach  to  these  problems.  It  is  centered  upon  the  test  of  the 
hypothesis  that  all  of  the  alternatives  are  equivalent.  Yet  the 
alternatives  are  actually  different  and  it  is  reasonable  to  expect  some 
difference  in  performance,  however  slight.  Thus,  the  failure  to  reject 
the  null  hypothesis  only  indicates  that  the  test  was  not  sufficiently 
powerful  to  detect  the  differences  -  e.g.,  a  longer  run  would  have  to  be 
employed.  Moreover,  even  when  the  investigator  rejects  the  hypothesis,  it 
is  highly  likely  that  he  is  more  interested  in  identifying  the  best 
alternatives  than  in  simply  concluding  that  the  alternatives  are  not 
equivalent.  Recently  proposed  ranking  procedures  .  .  .  seem  more 
appropriate  to  the  problem  than  the  conventional  analysis  of  variance 
techniques  .  .  . 


Multiple-ranking  procedures  can  not  only  handle  the  problem  of 
ranking  alternatives  and  selecting  among  them  based  on  empirical  results 
obtained  by  real-world  experiments  or  computer  simulation  experiments, 
but  also  can  design  an  experiment  in  advance  and  determine  the  sample 
size.  The  easiest  way  of  ranking  the  set  of  policies  is  to  rank  the  sample 
means  associated  with  the  given  policies.  But  sample  rankings  may  result 
in  incorrect  rankings  because  of  random  errors.  Given  a  specified  desired 
probability  of  correct  selection,  multiple-ranking  procedures  can 
correctly  rank  the  given  policies  with  the  desired  probability  whenever 
the  difference  between  the  highest  and  the  second  highest  population  means 
is  greater  than  a  specified  value. 

Bechhofer  [1054],  has  developed  a  single-sample  procedure  for 
ranking  means  of  normal  populations  with  known  variances .  Dunnett  [i960] 
and  others  also  discussed  ranking  procedures  under  the  assumption  of  known 
and  equal  variances.  Zinger  and  St.  Pierre  [1958]  considered  the  case  of 
unequal  but  known  variances. 

A  two-sample  (i.e.,  two-stage)  procedure  was  first  proposed  by  Stein 
[1945]  in  the  case  of  one  population.  Bechhofer,  Dunnett  and  Sobel  [1954] 
applied  a  two-sample  Stein-type  solution  for  the  case  of  unknown  but  equal 
variances.  Kleijnen  and  Naylor  [1969]  discussed  some  possible 
approximate  procedures  for  the  cases  of  known  and  unequal,  or  unknown  and 
equal,  variances.  They  "...  warn  the  reader  against  the 
indiscriminate  use  of  these  procedures  in  cases  where  the  variance  in 
unknown  but  has  been  estimated"  (p.  610),  and  note  that  "further  empirical 
research  is  definitely  needed  in  order  to  properly  evaluate  these 
approximative  techniques"  (p.  613). 
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Dudewicz  [1971]  has  proved  that  it  is  impossible  to  use  a  single- 
sample  procedure  (i.e.,  a  procedure  with  a  fixed  size  sample  or  number  of 
replicate  runs)  which  has  a  probability  of  correct  selection  that  is 
independent  of  the  variances.  Recent  work  oi  Dudewicz  [1972]  and  Dudewicz 
and  Dalai  [1975]  allows  for  unknown  and  unequal  variances  by  using  a  two- 
sample  procedure.  A  description  and  the  discussion  of  the  superiority  of 
this  procedure  is  presented  in  the  next  section.  Their  procedure  is 
suitable  for  simulations  of  business  and  accounting  systems  because  most 
of  the  population  variances  of  these  models  are  unknown  aaa  unequal. 

A  HETEROSCEDASTIC  MULTIPLE-RANKING  PROCEDURE 


In  business  and  accounting  simulation  applications,  one  assumes  the 
unknown  population  variances  are  equal  only  due  to  lack  of  procedures 
which  can  handle  the  case  of  unequal  unknown  variances.  For  example, 
Kleijnen  and  Naylor  [1969,  pp.  609-610]  stated  that,  "Only  in  rare  cases 
can  the  assumption  of  a  common  known  variance  be  expected  to  hold  with 
computer  simulation  experiments  with  models  of  business  and  economic 
systems . " 

The  aforementioned  Dudewicz  [1972]  and  Dudewicz  and  Dalai  [1975]  can 
be  specified  as  follows: 

Assume  there  are  k  populations  or  policies  (denoted  by  n  n  ... 

-L  ♦  >  > 

71.)  under  consideration,  and  that  n.  yields  observations  which  are 

2 

normally  distributed  with  unknown  mean  u.  and  unknown  variance  a. 

’  l  i 

(i  =  1,  2,...,k).  The  experimenter  wants  to  take  that  number  of 
observations  from  each  population  such  that  the  probability  of  correct 
selection  P(CS)  is  at  least  P*  (1/k  <  P*  <  1)  if  the  experimenter  requires 
the  best  mean  better  than  the  next  best  by  at  least  6*  (0  <  6*),  i.e., 


he/she  needs  to  have 


P(CS)  >  P*  whenever  >  <S* 

where  p  j-^j  and  are  population  means  for  the  best  and  the  next  best 

populations,  respectively.  For  example,  the  decision-maker  desiring  to 
select  the  best  sales  plan  among  K  plans ;  may  wish  to  be  95%  sure 
(P*  =0.95  here)  of  a  correct  selection  if  the  best  plan  is  better  than  the 
next  best  by  $100  (6*  =  $100  here)  or  more.  (Note  that  6*  and  P*  are  sub¬ 
jective  choices  by  the  decision-maker  depending  on  his/her  knowledge  and 
experience  on  the  problem  of  interest.)  The  procedure  has  the  following 
four  steps: 

Step  1:  Take  an  initial  sample  X.,.  .....  X.  of  size 

r  r  ll  m0 

a  (>  2)  from  population  i  (rt,  ),  and  calculate  the 
sample  mean 


1° 

X.  (n  )  =  *  X.  . 
i  o  j=l  1J 


X.  ./n 


and  sample  variance 

C  =  jn  (xu  •  (V»  V 


l) 


Step  2:  Set  the  sample  size  or  number  of  replicate  runs  by 


°i  =  max  *n0  +  1>  Up-  '  S.)2] 


Steo  3: 


where  [y]  denotes  the  smallest  integer  >  y,  (i  =  1,  2, 

...,  k) .  For  example,  [4.3]  =  5,  [6.9]  =  7,  [4j  =  4 
(since  the  sample  size  must  be  an  integer),  h  is  the 
percentage  point  which  was  tabulated  by  Dudewicz,  Ramberg, 
and  Chen  [1975]. 

Take  (n.  -  n  )  additional  observations 
r  o 

X.  _  ,  X.  , « ,  ...  ,  X.  from  the  population  fl .  , 
l ,  n  +1  j.,n+2  l ,  n .  x 

’  c  ’  o  i 
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and  calculate 


n . 

i 


Y,(n.  -  n  ) 


1  1 


jin  ♦  1  Xij/llli  '  no)  (i  =  l*  2 -  k) 


X.  =  b.  X.  (n  )  ♦  (1-b.)  Y.  (n.  -  n  ) 
i  l  i  o  i.  © 


where  b . 

l 


n. 

L 


(l 


a .  -  n 


H - t 

q  k  ,  h  .  ,2 
0 


)) 


(i  =  1,  2,  k) 

X^  (i  =  1,  2,...,  k)  are  estimated  population  means. 


_  Step  4:  JHank  order  X.  and  select  the  population  with  the  highest 
of  Xv  .  .  Xk.  1 

In  essence,  this  multiple  ranking  procedure  uses  a  two-stage 
sampling  process  to  compute  the  estimated  population  means,  then  ranks  and 
selects  the  population  with  the  highest  estimated  population  mean. 

This  procedure  is  superior  to  any  single-stage  procedure  largely  be¬ 
cause  the  probability  of  correct  selection  is  independent  of  the  unknown 
variances.  This  makes  it  possible  to  evaluate  applicable  percentage  points 
in  practice  while  with  a  single-stage  procedure  it  is  impossible  to  have  the 
probability  of  correction  selection  independent  of  the  population 
variances,  hence  the  probability  of  correct  selection  with  a  single-stage 
procedure  is  unknown  and  such  procedures  are  useless.  This  result  has 
been  stated  as  a  theorem  by  Dudewicz  and  Dalai  [1975] . 

To  illustrate  these  results  this  paper  applies  Dudewicz  and  Dalai’s 
multiple-ranking  procedure  to  a  simulation  of  multiple  objective 
production  planning  and  accounting  control  systems. 
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AN  ILLUSTRATIVE  MODEL 


With  the  empirical  evidence  that  firms  do  not  have  a  single  objective 

of  profit  maximization,  accountants  cannot  ignore  multiple  objective 

decision  problems.  This  paper  assumes  a  hypothetical  firm  which 

manufactures  and  sells  two  different  products,  and  which  has  two 

objectives  of  profit  and  sales.  The  two  product-mix  planning  models 

developed  are  a  satisficing  goal  programming  model  and  an  optimizing 

multiple  objective  linear  programming  model.  For  performance  evaluation, 

a  traditional  standard  cost  accounting  variance  analysis  and  an  ex  post 

accounting  variance  analysis  are  used  as  two  accounting  control  systems.^ 

The  main  objective  of  the  study  is  to  identify  which  of  the  four 

combinations  of  planning  models  and  accounting  variance  analysis  systems 

results  in  the  highest  mean  value  with  respect  to  actual  profit  and  actual 

sales  per  period.  Using  the  level  of  aspiration  concept,  standard 

resource  input  prices  and  quantities  are  revised  each  period.  The  actual 

performance  is  assumed  to  be  affected  by  the  level  of  aspiration  and  other 

unknown  random  behavioral  variables.  Level  of  aspiration  is  assumed  to  be 

2 

stochastically  related  to  past  performance  and  the  new  planned  budget. 

The  overall  model  of  the  firm  is  summarized  in  Figure  1.  This  is  a 
complex  model.  It  incorporates  behavioral,  level  of  aspiration  concepts, 
quantitative  structured  mathematical  programming  techniques  with  multiple 
objectives,  and  two  types  of  accounting  variance  analysis  systems.  The 
management  cycle  of  the  firm  is  divided  into  the  following  planning, 
operation  and  performance  evaluation  processes. 
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INSERT  FIGURE  I 


The  Planning  Process 

The  planning  process  includes  a  resource  allocation  decision  which 
is  a  multiple  objective,  uncertain  demand,  production  planning  model  with 
labor,  warehouse  and  machine  capacity  constraints.  It  fits  a  modified 
product-mix  model  since  it  adds  the  following  characteristics  to  a  pure 
product-mix  formulation:  (1)  considers  uncertain  demand,  (2)  incor¬ 
porates  inventory  level,  and  (3)  allows  overtime,  idle  time,  firing,  and 
hiring  labor.  Two  alternative  models,  a  satisficing  goal  programming  with 
uncertain  demand  and  an  optimizing  multiple  objective  linear  programming 
with  uncertain  demand,  are  formulated.  Both  models  have  the  same 
constraints  such  as  demand,  machine  capacity,  labor  resource,  overtime 
limits,  and  inventory  warehouse  capacity. 

The  goal  programming  model  has  two  additional  goal  constraints  for 
both  profit  and  sales  targets.  The  objective  function  is  to  minimize  the 
penalty  weights  on  deviations  from  the  profit  and  sales  targets.  The 
solution  used  Lee's  [1972]  modified  simplex  algorithm  to  obtain  planned 
production,  inventory  and  sales  quantities. 

The  multiple  objective  linear  programming  model  is  a  vector  maxi¬ 
mization  model  with  profit  and  sales  objective  functions.  Belenson  and 
Kepur's  [1973]  game  theory  algorithm  was  used  to  solve  this  model. 

The  Operations  Process 

Since  there  is  no  real  decision  maker  involved  in  this  study,  the 
operations  process  is  based  on  a  series  of  assumptions  and  decision  rules 
similar  to  Demski  [1971], 
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The  actual  demand  quantity  is  assumed  to  be  normally  distributed  with 
a  mean  which  equals  the  forecasted  sales  and  a  pre-specified  standard 
deviation.  The  actual  price  is  derived  by  a  function  similar  to  the 
forecasted  price  function  with  some  information  distortion.  The  act  ;1 
material  price,  material  usage  per  output  unit,  labor  wage  rate,  laL or 
hours  per  output  unit,  variable  overhead  rate,  and  machine  hours  ngr 
output  unit  are  assumed  to  be  normally  distributed  with  a  mean  equal  to 
the  standard  amount  adjusted  for  the  level  of  aspiration  change.  The 
actual  production  quantity  is  a  linear  function  of  the  planned  quantity 
and  a  behavior  implementation  variation. 

The  Performance  Evaluation  Process 

The  performance  evaluation  process  includes  an  accounting  variance 
analysis  and  a  transformation  function.  The  accounting  variance  analysis 
compares  actual  results  to  the  standards.  There  are  two  approaches  to 
accounting  variance  analysis:  (1)  traditional  standard  cost  variance 
analysis,  and  (2)  Demski's  [1967]  ex  post  variance  analysis. 

A.  Traditional  Variance  Analysis 

Traditional  standard  cost  variance  analysis  compares  an  ex  ante 
budget,  a  budget  adjusted  to  the  actual  activity  level,  and  actual 
results: 

Ex  Ante  Profit  -  Actual  Profit  =  (Ex  Ante  Profit  -  Adjusted  Profit) 

+  (Adjusted  Profit  -  Actual  Profit) 

=  Mix  and  Volume  Variance  +  Price  and  Efficiency  Variance 
+  Fixed  Cost  Variance 

The  price  and  efficiency  variance  can  be  divided  into  a  sales  price 
variance,  a  materials  price  variance,  a  materials  usage  variance,  a  labor 


wage  rate  variance,  a  laboc  efficiency  variance,  a  variable  overhead 

spending  variance,  and  a  variable  overhead  efficiency  variance. 

3 

B.  Ex  Post  Variance 

Demski's  ex  post  variance  analysis  can  be  expressed  as  follows: 

Ex  Ante  Profit  -  Actual  Profit  =  (Ex  Ante  Profit  -  Ex  Post  Profit) 

+  (Ex  Post  Profit  -  Actual  Profit) 

=  forecasting  Variance  +  Opportunity  Cost  Variance 
+  Fixed  Cost  Variance 

Opportunity  cost  variance  can  be  divided  into  a  mixed  and  volume 
variance,  and  a  price  efficiency  variance. 

The  transformation  function  is  a  stochastic  process  to  characterize 
t'-e  influences  of  this  period's  ex  ante  planned  profits,  sales,  resource 
price,  and  quantity  standards  together  with  last  period's  accounting 
variance  analysis  results  and  the  weighted  average  of  past  performance  on 
this  period's  actual  production  quantities,  resource  prices  and 
quantities . 

There  are  two  factors  in  this  study:  planning  and  accounting 
variance  analysis.  Response  variables  are  average  actual  profits  per 
period  and  average  actual  sales  per  period.  The  four  treatment 
combinations  are: 

Mode  1  -  using  goal  programming  planning  model  and 
traditional  variance  analysis  model 

SIMULATION  ANALYSIS 

With  simulation,  the  decision-maker  can  examine  a  model  under  a  variety 

of  contingencies.  In  this  way,  the  model  can  be  analyzed  without 
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costly  experimentation  of  field  or  case  studies. 


Mode  2  -  ising  goal  programing  planning  model  and 
ex  post  variance  analysis  model 

Mode  3  -  using  multiple  objective  linear  programming 

planning  model  and  traditional  variance  analysis 
model 

Mode  4  -  using  multiple  objective  linear  programming 
planning  model  and  ex  post  variance  analysis 
model 

Since  the  decision  is  production  planning,  a  two  year  run  is 
sufficient  for  the  manager  to  make  short-run  decisions.  The  planning 
period  is  assumed  to  be  a  month.  For  each  sample  (i.e.,  replication  run), 
the  total  run  length  is  twenty-four  periods. 

Following  is  the  simulation  result  of  Dudewicz's  and  Dalai's  [1975] 
two-stage  sampling  procedure: 

Step  1:  Take  an  initial  sample  of  size  30  replications  from  mode  i 

(n^)  and  calculate  sample  means  and  sample  variances  for 
actual  profit  per  period,  and  actual  sales  per  period, 
respectively. 

The  sample  means  and  variances  for  two  response  variables  and  30 
replications  are  shown  in  Table  1. 
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Table 

1.  Initial 

Sample 

Means  and 

Variances 

(nQ  = 

30) 

Variables 

Mode  1 

Mode  2 

Mode  3 

Mode  4 

Actual  Profit 

Mean 

742 

792 

312 

812 

Per  Period: 

Variance 

12309 

11794 

10078 

10096 

Actual  Sales 

Mean 

6220 

6364 

6539 

6537 

Per  Period: 

Variance 

386S2 

33591 

39816 

37998 

where: 

Mode  1  = 

Mode  2  = 
Mode  3  = 

Mode  4  = 

Step  2: 


Goal  Programming  with  Traditional  Variance 
Analysis 

Goal  Programming  with  Ex  Post  Variance  Analysis 

Multiple  Objective  Linear  Programming  with 
Traditional  Variance  Analysis 

Multiple  Objective  Linear  Programming  with 
Ex  Post  Variance  Analysis 

For  the  multiple-ranking  procedures,  the  sample 

size  of  number  of  replicate  runs  (n^)  for  each 

mode  i  is  determined  by 

n.  =  max  n.  .  =  max  max  {31,  [*r-  S. .  )^1} 
i  ij  ‘  ‘•o-  ij  “ 

J  J  J  J 

where  j  =  the  response  variable  number,  j  “  1,  2 

ic 

h,  6  and  S. .  are  described  in  the  previous  section. 
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From  the  Dudewicz,  Ramberg  and  Chen  [1975]  tables,  if  the  least 
sample  difference  to  be  detected  for  all  response  variables  is  taken  to  be 
as  in  Table  2,  and  the  confidence  level  is  at  least  95%,  h  is  3.03  for  4- 
mode  comparisons  or  rankings.  By  this  choice,  we  can  be  sure  that  we  are 
95%  correct  in  selecting  the  best  alternative  in  Table  3. 


Table  2.  Specified  Value  of  the  Least  Sample  Difference 


Variables 

The  Least  Sample  Difference  6* 

Actual  Profit  Per  Period 

$30,  or  approximately  4%  of  the 
average  profit 

Actual  Sales  Per  Period 

$100,  or  approximately  1.5%  cf 
the  average  sales 

Table  3.  Number  of  Replicate  Runs  (n^: 
Four-Mode  Rankings 


Variables 

Mode  1 

Mode  2 

Mode  3 

Mode  4 

Actual  Profit  Per 
Period 

126 

121 

103 

103 

Actual  Sales  Per 
Period 

36 

31 

37 

35 

n.  =  max  n. . 
i  ij 

j 

126 

121 

103 

103 

Step  3:  Take  (n^  -  30)  additional  reriicate  runs  from  mode  i  and 

compute  the  generalized  sample  mean  X^  which  is  shown  in 
Table  4. 

Step  4  :  Select  the  mode  with  the  highest  . 
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(p* 

Table  4.  Multiple-Ranking 
=  95%,  nT  =  126,  n2  =  121, 

and  Selection 
n3  =  103,  n4  =  103) 

1. 

1 

Actual  Profit 

Per  Period 

Actual  Sales 

Per  Period 

Mode  1 

$749 

$6214 

Mode  2 

779 

6363 

Mode  3 

801 

6533 

Mode  4 

804 

6532 

Highest 

Mode  4 

Mode  3 

The  data 

in  Table  4  show  that  there 

is  not  much  difference 

mode  3  and  4  either  in  terms  of  the  actual  profit  or  the  sales  per  period. 
Mode  4  ranks  highest  in  the  actual  profit  per  period,  while  mode  3  ranks 
highest  in  the  actual  sales  per  period. 
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Since  mode  3  and  4  are  very  close,  a  confidence  interval  on  the 
difference  between  mode  3  and  4  for  the  mean  actual  profit  and  sales  per 
period  is  estimated  by  Dudewicz,  Ramberg  and  Chen's  [1975]  multiple 
comparison  procedure. 

Given  k  =  2  (number  of  modes  for  comparison),  n^  =  n4  =  126,  and 
assuming  that  mode  3  is  a  control  mode,  one  can  compute  and  by  using 
the  first  three  steps  of  multiple-ranking  procedure.  Then  the  estimated 
confidence  intervals  are: 

For  upper  intervals,  state  that 
M4  -  M3  <  -  X,)  +  5* 

or  for  lower  intervals,  state  that 
(f4  -  S3)  -  6*  <M4  -  H3 

All  symbols  are  defined  previously.  The  estimated  confidence 
intervals  are  presented  in  Table  6.  (1  -  a)  is  derived,  from  Dudewicz, 

Ramberg  and  Chen's  Table  [1975]. 


Table  6.  Confidence  Intervals  on  the  Difference 
Between  Mode  3  and  Mode  4 
(1  -  a  =  95%  n3  =  n4  =  126) 


Actual  Profit 

Actual  Sales 

Difference 

Per  Period 

Per  Period 

Prespecified 
Difference  6* 

$30 

$100 

m4  ’  m3  > 

$  -28.03 
(-3.5%) 

$  -100.80 
(-1.5%) 

or 

^4  *  < 

$  31.97 

(4.0%) 

$  99.20 

(1.5%) 
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The  data  iu  Table  6  shows  that  there  is  not  much  difference  between 
mode  3  and  4.  Mode  4  is  at  most  $28.03  Lower  or  at  most  $31.9?  higher  than 
mode  3  in  actual  profit  per  period  with  a  95%  confidence  and  mode  4  is  at 
most  $100.80  lower  or  at  most  $99.20  higher  than  mode  3  in  actual  sales 
per  period  with  a  95%  confidence. 

In  summary,  the  results  of  simulation  analysis  with  a  probability  of 
95%  correct  selection  for  the  true  ranking  of  the  population  means  are  as 
follows : 

1.  For  the  simulated  profit  per  period,  the  ranking  sequence  is: 

Mode  4  >  Mode  3  >  Mode  2  >  Mode  1  That  is,  the  rankings  are: 
(1)  Multiple  Objective  Linear  Programming  with  Ex  Post  Variance 
Analysis,  (2)  Multiple  Objective  Linear  Programming  with 
Traditional  Variance  Analysis,  (3)  Goal  Programming  with  Ex 
Post  Variance  Analysis,  and  (4)  Goal  Programming  with 
Traditional  Variance  Analysis. 

2.  For  the  simulated  sales  per  period,  the  ranking  sequence  is: 

Mode  3  >  Mode  4  >  Mode  2  >  Mode  1 

The  theory  for  multivariate  multiple-comparisons  and  ranking 
procedures  has  not  yet  been  developed.  The  results  shown  in  this  research 
are  independent  comparisons  and  rankings  for  each  response  variable. 
Given  the  data  in  Table  4,  mode  4  is  the  best  for  profit  and  mode  3  is  the 
best  for  sales.  There  is  no  conclusion  drawn  on  which  mode  (mode  3  or  4) 
should  be  selected  with  respect  to  the  two  objectives  when  considered 
together. 
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CONCLUSIONS 


With  the  aid  of  a  multiple  objective  firm  example,  we  have  attempted 
to  demonstrate  an  improved  multiple-ranking  procedure  to  analyze 
simulated  data  which  explicitly  accounts  for  unknown  and  unequal 
variances.  In  accounting  or  business  system  simulations,  the 
experimenter  usually  has  two  objectives  in  mind:  (1)  to  test  whether 
different  system  population  means  are  equal  or  not;  if  not,  a  pairwise 
comparison  may  be  used  to  find  out  which  one  is  responsible  for  the 
rejection,  and  (2)  to  rank  and  select  the  best  system  from  all  the  systems 
examined.  However,  the  former  F-statistics  only  tell  us  whether  there  is 
a  significant  difference  between  two  policies  (i.e.,  to  satisfy  the  first 
objective)  under  equal  variance  assumption.  The  latter,  for  a  specified 
degree  of  certainty,  indicates  the  best  policy  (i.e.,  to  satisfy  the 
second  objective).  Whea  variances  are  unequal  (unknown),  the  two-stage 
procedure  is  the  only  available  procedure  to  reach  a  possible  solution. 

This  paper  illustrates  a  two-stage  sampling  multiple-ranking 
procedure  which  has  four  steps.  The  first  step  is  to  take  an  initial 
sample  size  from  each  population  and  compute  sample  mean  and  variance. 
The  second  step  is  to  determine  optimal  sample  size  by  considering  the 
experimenter's  minimum  probability  of  correct  selection  and  his/her 
specified  minimum  difference  between  the  best  and  next  best  population. 
The  third  step  is  to  take  an  additional  sample,  which  is  the  difference 
between  the  optimal  sample  size  and  the  initial  sample  size,  and  compute 
estimated  population  means.  The  last  step  is  to  make  a  selection  based  on 
the  estimated  population  means. 

From  the  results  of  the  multiple-ranking  procedure  based  on  the 
simulated  output  data,  one  may  conclude  that  the  more  costly  ex  post 
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variance  analysis  seems  indistinguishable  from  traditional  variance 
analysis,  both  on  sales  and  profit,  if  multiple  objective  linear 
programming  is  used  (while  there  is  a  difference  if  goal  programing  is 
used).  Since  multiple  objective  linear  programming  turned  out  better  than 
goal  programming,  one  can  use  cheaper  traditional  accounting  variance 
analysis  with  multiple  objective  linear  programming  to  generate  higher 
sales  and  profit. 

The  multiple- ranking  procedure  can  be  applied  to  various  accounting 
and  business  issues.  For  example,  the  LIFO  vs.  FIFO  inventory  costing 
methods  can  be  considered  together  with  the  straight-line  vs  -  double- 
declining  balance  depreciation  methods  for  a  hypothetical  firm.  The  best 
possible  combination,  in  terms  of  the  highest  mean  profit  or  other 
criteria,  can  be  selected.  One  also  can  apply  this  procedure  to  select 
the  best  job-shop  scheduling  rule  or  queueing  algorithm.  The  reader  is 
cautioned,  however,  to  avoid  using  these  techniques  without  regard  for  the 
two  assumptions  of  independence  and  normality. 
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FOOTNOTES 


1.  Traditional  accounting  variance  analysis  compares  an  ex  ante  planned 
budget,  a  flexible  budget  which  is  adjusted  to  the  actual  activity 
level,  and  actual  results.  Ex  post  accounting  variance  analysis 
compares  an  ex  ante  planned  budget,  an  ex  post  optimal  budget,  and 
actual  results.  Ac  ex  post  optimal  budget  is  the  one  a  firm  would 
have  used  to  determine  the  ex  ante  budget  had  it  forecast  all  model 
parameters  correctly. 

2.  A  more  detailed  description  of  the  ten  major  behavioral  assumptions 
and  the  planning  and  control  processes  of  the  simulated  firm  is  given 
in  Lin  [1978] . 

3.  Lin  [1980]  developed  and  illustrated  models  of  ex  post  analysis  under 
both  goal  programming  and  multiple  objective  linear  programming. 

4.  The  choice  of  an  initial  sample  size  is  based  on  the  discussion  of 
the  paper  by  Bechhofer,  Dunnett  and  Sobel  [1954].  The  choice  of  30 
initial  sample  size  is  appropriate  in  this  case. 
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